Skip to content

test(js-sdk): run the full unit test suite in a browser - #1609

Open
mishushakov wants to merge 5 commits into
mainfrom
browser-test-suite
Open

test(js-sdk): run the full unit test suite in a browser#1609
mishushakov wants to merge 5 commits into
mainfrom
browser-test-suite

Conversation

@mishushakov

@mishushakov mishushakov commented Jul 24, 2026

Copy link
Copy Markdown
Member

Important

Blocked on one production deploy. The SDK side is already un-gated, so no further code change is needed here — the browser leg just stays red until it rolls out (the leg only ever runs against production; staging callers pass node-only, which drops it).

needs unblocks symptom until then
infra#3388 6 Sandbox.list({ limit }) pagination tests rolled out — these pass now
infra#3389 (open) 15 tests: 14 stopped-sandbox + the traffic-access-token one 14 × TypeError: Failed to fetch, 1 × SandboxError instead of TimeoutError

Everything else is green: 8 failed files, 65 passed, 2 skipped — and every one of the 15 failing tests is in the table above.

What

Promotes the browser leg from a single React smoke test to the full unit + connectionConfig suite running inside a real headless Chromium (@vitest/browser + Playwright) — the same coverage test:bun / test:deno / test:cf get. 69 files / 343 tests green locally against production.

pnpm test:browser   # installs Chromium via pretest:browser, then runs the suite

React is gone as requested — the smoke test (tests/runtimes/browser/run.test.tsx) and its seven devDependencies (react, react-dom, @types/react, @types/react-dom, @testing-library/react, @vitejs/plugin-react, vitest-browser-react). Chromium installation moves off the default pretest hook onto pretest:browser, so plain pnpm test no longer downloads a browser.

The one thing the smoke test uniquely covered — the SDK driven with no process global at all, the way a real browser bundle runs — is kept as a dedicated test, since the rest of the suite runs against a process.env shim that would mask a regression there:

// tests/runtimes/browser/noProcessGlobal.test.ts
Reflect.deleteProperty(globalThis, 'process')
const sandbox = await Sandbox.create(template, { apiKey, domain })
await sandbox.files.write('hello.txt', 'Hello World')

Failure screenshots are off (browser.screenshotFailures: false): these tests never render anything, so vitest's default would write a PNG of a blank page per failed test into .vitest-attachments/.

Test bugs the leg surfaced

  1. Two assertions were vacuous. commands/kill and pty/kill asserted rejects.toThrowError(ProcessExitError) — but ProcessExitError has never been exported from src. Vite's SSR transform resolves the missing named import to undefined, so toThrowError(undefined) asserted nothing on any runtime; real ESM in the browser turns it into a hard SyntaxError. Now CommandExitError.

  2. files/read.test.ts ran three tests twice — a byte-identical duplicated block, provisioning a second real sandbox for each on every runtime leg.

  3. A cold-cache-only flake, root-caused. api/http2 and envd/http2 carried dead vi.doUnmock('undici') / vi.doUnmock('../../src/utils') calls from an old refactor (neither file mocks anything any more). The bare 'undici' one made Vite discover and pre-bundle a Node HTTP client mid-run, reloading the browser page under the running suite and breaking whichever file was being collected. Verified: 1-in-4 failure before, three cold-cache runs green after.

Node-only test APIs are replaced with cross-runtime equivalents: node:crypto randomUUID/createHash → WebCrypto, BufferTextEncoder/TextDecoder, path.basename → inline.

Two real browser bugs found

Both verified over the wire; neither was fixable in the SDK alone.

Pagination silently truncated — the cursor lives in the x-next-token response header, which the API didn't name in Access-Control-Expose-Headers, so a browser withheld it and every page looked like the last:

const paginator = Sandbox.list({ limit: 10 })
const all = []
while (paginator.hasNext) all.push(...(await paginator.nextItems()))
// Node: all 200 sandboxes.  Browser (before the fix): 10, hasNext === false, no error.

Fixed upstream in infra#3388, so the tests are un-gated here and just need the prod rollout (see the callout above). SDK-293.

SDK-294 — isRunning() threw instead of returning false. A stopped sandbox's 502 came from the edge without CORS headers, so the probe rejected opaquely (running: 204 + ACAO: * / paused: 502 + no ACAO). Because a browser always preflights the SDK's envd requests (they carry E2b-Sandbox-Id/E2b-Sandbox-Port) and a preflight needs an ok status, adding the header to the 502 alone wasn't enough — the OPTIONS path needed a 2xx too. This also silently downgraded the kill-mid-request error from the actionable TimeoutError to a generic SandboxError across 24 call sites.

Fixed upstream in infra#3389, which also fixes the content negotiation the investigation turned up: the proxy chose HTML over JSON by User-Agent sniffing, and browser fetch can't override its UA, so an SDK call from a browser got the HTML error page where it expects JSON (Accept was ignored). Both are un-gated here.

How limitations are handled

No capability flags — nothing is skipped for being a browser. The five tests that read a response from a server the test itself starts in the sandbox now start a CORS-enabled one, via corsHttpServerCmd in tests/setup.ts:

// was: 'python3 -m http.server 8000' — no CORS headers, so a browser hands the
// test an opaque `TypeError: Failed to fetch` instead of the response
await sandbox.commands.run(corsHttpServerCmd(8000), { background: true })

This was originally gated behind a canFetchSandboxServers flag as an inherent browser limitation. It isn't one: the server is the test's own, and a real browser app's server opts into CORS exactly the same way — so the gate was hiding coverage of getHost and the proxy path rather than documenting a wall. Access-Control-Allow-Headers plus do_OPTIONS also covers the traffic-access-token test, whose custom header the browser preflights.

Confirmed on the browser leg (run): of the five tests this un-gates, three now passping server in running sandbox, sandbox works without token, auto-resume wakes paused sandbox on http request. That's the CORS server working end-to-end through the proxy against production. The other two are the two infra#3389 needs anyway: pause and resume a sandbox with http server calls isRunning() on a paused sandbox, and sandbox requires traffic access token reads a proxy-synthesized 403 (which 3389 gives CORS headers) after a preflight that can't carry the token (which 3389 answers on the getDestination error path).

The exclude list is only for genuinely Node-only suites (tests/bundle/** reads dist via node:fs, tests/undici.test.ts resolves packages off process.versions.node) plus the two msw suites, whose msw/node entry pulls in node:http and can't be served to a browser. Porting those needs setupWorker plus a service worker from a public dir — tracked as a follow-up on SDK-292, and worth doing since cancellation is the most runtime-divergent part of this SDK.

CI

New browser matrix leg on ubuntu-22.04, with the Playwright cache moved onto it (no other leg needs Chromium any more). The leg is dropped by node-only staging callers, and it's the only leg whose result depends on the API's CORS headers — the matrix comment now says so, rather than claiming the non-Node legs add no backend signal.

Notes

  • No changeset: test-only, matching test(js-sdk): run the template test suite on Bun #1600 and the other recent test(js-sdk) commits.
  • vi.stubEnv and delete process.env.X keep working in the browser because the shim aliases process.env to import.meta.env, which is where vitest puts config env and where stubEnv writes. Verified E2B_DEBUG=1 pnpm test:browser correctly skips the debug-gated tests.
  • Verification for the CORS server, since a browser-side change like this is easy to get subtly wrong: ran a local instance of the exact generated command and fetched it cross-origin (different port = different origin) from real Chromium — simple GET readable, and a custom-header request survives the preflight. Negative control against a plain python -m http.server reproduces the TypeError: Failed to fetch the browser leg was showing, so the probe is meaningful rather than vacuous. Also checked there's exactly one Access-Control-Allow-Origin header — a duplicate would make browsers reject the response.
  • Review feedback addressed in 4c94fca: the pause and resume a sandbox with http server test now carries the canFetchSandboxServers gate too (it fetches its own python -m http.server, so it was a real miss that the isRunning failure happened to mask), and noProcessGlobal.test.ts imports the SDK dynamically inside the test body — a static import is evaluated at collection time, while the process shim is still installed, so it only covered the call path and not module evaluation.
  • No new type errors: 22 pre-existing test type errors before and after (the 2 that went away are the ProcessExitError ones this fixes).
  • Not addressed, pre-existing, flagged in case you want a follow-up: tests/api/list.test.ts duplicates five tests byte-for-byte (only the three betaPause ones intentionally differ, instance vs static), and files/write.test.ts:130 repeats write file as a strict subset of the test at :6 — together ~7 redundant sandboxes per run per leg. Left alone rather than deleting tests outside this change's scope.

Closes SDK-292

🤖 Generated with Claude Code

Promotes the browser leg from a single React smoke test to the full unit +
connectionConfig suite running inside a real headless Chromium via
@vitest/browser + Playwright (`pnpm test:browser`) — the same coverage
test:bun / test:deno / test:cf get. 69 files / 343 tests green.

The React smoke test and its devDependencies are gone (react, react-dom,
@types/react{,-dom}, @testing-library/react, @vitejs/plugin-react,
vitest-browser-react). The one thing it uniquely covered — the SDK driven with
no `process` global at all, as in a real browser bundle — is kept as
tests/runtimes/browser/noProcessGlobal.test.ts. Chromium installation moves
from the default `pretest` hook to `pretest:browser`, so plain `pnpm test` no
longer downloads a browser.

Test bugs the leg surfaced:

- commands/kill and pty/kill asserted `rejects.toThrowError(ProcessExitError)`,
  but `ProcessExitError` is not exported from src. Vite's SSR transform
  resolved the named import to `undefined`, so both assertions were vacuous;
  real ESM in the browser makes it a hard SyntaxError. Now CommandExitError.
- files/read.test.ts had three byte-identical duplicated tests, each
  provisioning a second sandbox on every runtime.
- api/http2 and envd/http2 carried dead `vi.doUnmock('undici')` /
  `vi.doUnmock('../../src/utils')` calls from an old refactor. The bare
  'undici' one made Vite pre-bundle a Node HTTP client mid-run and reload the
  page under the running suite — a cold-cache-only flake that broke whichever
  file was being collected at the time.

Node-only test APIs are replaced with cross-runtime equivalents: node:crypto
randomUUID/createHash to WebCrypto, Buffer to TextEncoder/TextDecoder,
path.basename inlined.

Limitations a browser physically can't work around are gated by documented
capability flags in tests/setup.ts, so they report as skipped instead of
disappearing into the config's exclude list. Two are real user-facing bugs,
filed as SDK-293 (x-next-token is not in Access-Control-Expose-Headers, so
Sandbox.list pagination silently truncates) and SDK-294 (the stopped-sandbox
502 has no CORS headers, so isRunning() throws instead of returning false).

Closes SDK-292

Co-Authored-By: Claude <noreply@anthropic.com>
@linear-code

linear-code Bot commented Jul 24, 2026

Copy link
Copy Markdown

SDK-292

@cla-bot cla-bot Bot added the cla-signed label Jul 24, 2026
@cursor

cursor Bot commented Jul 24, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Test and CI wiring only—no production SDK behavior changes in this diff. Residual risk is flaky or red CI until production CORS fixes land, and heavier Playwright runs on the new leg.

Overview
Expands browser coverage from a single React smoke test to the same unit + connectionConfig suites that Bun/Deno/Cloudflare already run, executed in headless Chromium via @vitest/browser + Playwright (pnpm test:browser).

Adds a dedicated tests/runtimes/browser/vitest.config.mts with a process.envimport.meta.env shim, forwards only E2B_* env into the browser, and excludes Node-only suites (bundle/undici/msw-node). noProcessGlobal.test.ts dynamically imports the SDK with process removed so create/read/command paths work like a real bundle.

CI gains a browser matrix leg; Playwright install/cache moves off Node onto that leg. Staging node-only still drops browser (documented: only leg that exercises API CORS behavior).

Removes the old vitest browser React project, run.test.tsx, and React-related devDependencies; Chromium install moves from pretest to pretest:browser.

Shared test fixes for cross-runtime: CommandExitError instead of vacuous ProcessExitError asserts; dedupe in files/read.test.ts; drop stale vi.doUnmock('undici') in http2 tests; replace node:crypto/Buffer/path with Web APIs; corsHttpServerCmd for in-sandbox fetches; browser-aware skips/assertions for stable sandbox host / envdApiUrl.

Reviewed by Cursor Bugbot for commit bf5d7cf. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Package Artifacts

Built from 915c003. Download artifacts from this workflow run.

JS SDK (e2b@2.36.1-browser-test-suite.0):

npm install ./e2b-2.36.1-browser-test-suite.0.tgz

CLI (@e2b/cli@2.15.2-browser-test-suite.0):

npm install ./e2b-cli-2.15.2-browser-test-suite.0.tgz

Python SDK (e2b==2.35.0+browser.test.suite):

pip install ./e2b-2.35.0+browser.test.suite-py3-none-any.whl

Comment thread packages/js-sdk/tests/sandbox/snapshot.test.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5626cfff60

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@@ -0,0 +1,45 @@
import { assert, test } from 'vitest'

import { Sandbox } from '../../../src'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Import the SDK after removing the process shim

Because this static import is evaluated during test-file collection, after processEnv.setup.ts has already installed globalThis.process, this browser-only test no longer exercises SDK module evaluation in a real no-process environment. A regression that adds a bare top-level process read anywhere under ../../../src would pass here but still crash for browser users before Sandbox.create() runs; dynamically import the SDK inside the test after Reflect.deleteProperty(globalThis, 'process') (or run this file without the setup shim) to preserve the smoke test's original coverage.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and fixed in 4c94fca — the SDK is now imported inside the test body, after Reflect.deleteProperty(globalThis, "process"). As you say, the static import was evaluated at collection time with the shim still installed, so module-eval-time coverage was vacuous; only the call path was covered. Verified: the SDK module graph now evaluates with no process global and the test gets as far as Sandbox.create.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only PR that expands the browser CI leg to the full unit suite, with no production code changes.

Extended reasoning...

Overview

This PR touches only test infrastructure: the browser vitest config, test files (replacing Node-only APIs like Buffer/node:crypto with WebCrypto/TextEncoder), the CI workflow matrix, package.json devDependencies (dropping React test tooling), and pnpm-lock.yaml. No files under packages/js-sdk/src/ are modified.

Security risks

None. No auth, crypto business logic, or user-facing behavior changes — only test harness and CI configuration.

Level of scrutiny

Low-to-moderate scrutiny is appropriate: this is a test-only change with no production code path affected. I spot-checked the PR's own claims against src/: CommandExitError is indeed exported (ProcessExitError never existed, confirming the described vacuous-assertion bug fix), and the browser-specific skip flags (canFetchSandboxServers, canReadPaginationToken, canObserveStoppedSandbox) correctly mirror existing runtime !== 'browser' CORS-avoidance logic already present in src/connectionConfig.ts. The new capability-gated skips are a reasonable way to surface browser limitations without silently dropping coverage.

Other factors

The bug hunting system found no issues. The PR description is thorough and self-documents the rationale for each test change (duplicate test removal, flaky mock cleanup, Node-API replacements). No changeset is needed since this is test-only, consistent with prior similar PRs (#1600) per CLAUDE.md guidance.

These tests drive the SDK and never render anything, so vitest's
browser.screenshotFailures (on by default when headless) wrote a PNG of a blank
page per failed test into .vitest-attachments/. Turned off, which also lets the
.gitignore entry it needed go away.

Removes canReadPaginationToken: the API not exposing X-Next-Token via
Access-Control-Expose-Headers is fixed upstream in infra#3388, so the six
limit-based pagination tests are un-gated and run in the browser like
everywhere else.

Note infra#3388 is not on production yet, and the browser leg only ever runs
against production (staging callers pass node-only, which drops it), so those
six tests fail on `paginator.hasNext` until the deploy lands.

Co-Authored-By: Claude <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: bf5d7cf

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Comment thread packages/js-sdk/tests/setup.ts Outdated
Removes canObserveStoppedSandbox. The stopped-sandbox 502 arriving from the
edge without CORS headers is fixed in e2b-dev/infra#3389, so isRunning() can
report false in a browser again and checkSandboxHealth can return false, which
restores the actionable TimeoutError on the 24 call sites that use it.

Also un-gates host.test.ts 'ping server in non-running sandbox', which was
under the wrong flag: it asserts the edge's own 502 JSON envelope rather than a
response from a server inside the sandbox, so infra#3389 covers it too (the
same PR's content-negotiation fix is what keeps the body JSON rather than the
HTML error page). Its sibling 'ping server in running sandbox' does read the
user's python server, so that one keeps the gate.

canFetchSandboxServers stays for the four tests that read a response from a
server the test started in the sandbox — no CORS headers there, and one of them
sends a custom header that would preflight into python's 501. Its doc comment
no longer cites the proxy 502 as an example, since that now carries the header.

Until infra#3389 reaches production these 14 tests fail on the browser leg —
13 with 'TypeError: Failed to fetch' and one with the degraded SandboxError in
place of TimeoutError, matching the two documented consequences exactly. The
Node/Bun/Deno/workerd legs are unaffected (7 files / 26 tests green on Node).

Co-Authored-By: Claude <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit afe3fb8. Configure here.

Comment thread packages/js-sdk/tests/setup.ts Outdated
mishushakov and others added 2 commits July 25, 2026 11:14
- gate the http-server pause/resume test on canFetchSandboxServers: it
  fetches a `python -m http.server` the test starts itself, which sends no
  CORS headers. Masked today by the isRunning failure, it would have started
  failing the moment infra#3389 rolls out.
- import the SDK dynamically in noProcessGlobal, after the process shim is
  deleted. A static import is evaluated at collection time while the shim is
  still installed, so a top-level `process` read anywhere in the module graph
  would have passed here and still crashed a browser app on import.

Co-Authored-By: Claude <noreply@anthropic.com>
…browser gate

`python -m http.server` sends no CORS headers, so a browser handed the test an
opaque `TypeError: Failed to fetch` instead of the response. That was gated
behind `canFetchSandboxServers` as inherent, but it isn't: the server is ours,
and a real browser app's own server opts in the same way. `corsHttpServerCmd`
starts one that does, so the five tests run in Chromium too and the last
capability flag is gone.

Verified in Chromium against a local instance of the exact generated command:
cross-origin GET readable, and a custom-header request survives the preflight
via do_OPTIONS. Negative control against a plain `python -m http.server`
reproduces the `Failed to fetch` the browser leg was showing.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant